A Model Architecture for Mental Imagery

Imagination or mental imagery is the capability of generating sensory experiences without actual sensory inflow. These sensory experiences might be the results of actions that are not carried out physically but only simulated within the cognitive system of an agent. We propose a model architecture for mental imagery that enables an artificial agent to generate views of parts of himself based on a set of given pose parameters and according to a specific viewing direction. The architecture is evaluated in a robotic study with an agent consisting of a robotic manipulator and a camera head. The agent adaptively learns to associate views of its gripper with its current pose. Furthermore, a visual forward model predicts visual changes according to the agent’s gaze direction.